Repository navigation
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Critical runtime and HTTP defects allow jobs to exceed deadlines, interrupted promotion writes to prevent restart, and unauthenticated connections to exhaust server resources.
15 open issues
Enforce maxSeconds across training and evaluation · New Acquire exclusive state-directory ownership before recovery · New Enforce taskset digest checks for readiness, inference, and rollback · New Validate every question and enforce taskset option limits · New Publish promotion generations atomically · New Test runtime token validation and error redaction · New Limit concurrent connections and bound socket read times · New Prevent connection reuse when request bodies remain unread · New Use fixture backends only when backends is None · New Cancel queued jobs when closing the service · New Preserve rollback history when promoting the current candidate · New Move job setup inside the failure-handling try block · New Handle SIGTERM through the cleanup path · New Verify restart recovery leaves no candidate or promotion · New Document direct scoring requests instead of unavailable CLI commands · New
What changed in this PR
Adds a customer-operated local decision-model runtime to the learning package, extending #203’s taskset and evaluation work while keeping training and serving outside Hub.
Changes:
- Tracks asynchronous jobs, immutable candidates, evaluation reports, promotion, rollback, and restart recovery.
- Adds authenticated scoring, health endpoints, and a deterministic fixture backend with production restrictions.
- Includes a standalone container, lifecycle demo, runtime tests, and operator documentation.
| File | Description |
|---|---|
packages/learning/README.md |
Introduces and links to the local runtime. |
packages/learning/environments/brand-check/tests/test_runtime.py |
Covers lifecycle, persistence, authentication, and production restrictions. |
packages/learning/environments/brand-check/server/runtime.Dockerfile |
Packages the runtime in a non-root container. |
packages/learning/environments/brand-check/runtime.py |
Implements jobs, candidate storage, promotion, rollback, and inference. |
packages/learning/environments/brand-check/runtime_server.py |
Exposes authenticated runtime routes and health endpoints. |
packages/learning/environments/brand-check/README.md |
Documents operation, deployment, backend integration, and trust boundaries. |
packages/learning/environments/brand-check/pyproject.toml |
Registers the runtime command. |
packages/learning/environments/brand-check/examples/runtime_demo.py |
Demonstrates the complete fixture lifecycle. |
packages/learning/environments/brand-check/__init__.py |
Adds lazy exports for runtime classes. |
docs/adr/0002-decision-model-and-rl-environment.md |
Describes the local reference runtime and its limits. |
Review effort: Max (adaptive)
63954a1 to
3703c13
Compare
3703c13 to
3180e63
Compare
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
3180e63 to
82a6568
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical defects let timed-out training continue, allow state writes after ownership is released, apply failed promotions after restart, and permit slow header reads to exhaust request capacity.
6 open issues
Reconcile pending generations before promotion and rollback reads · New Terminate timed-out training before cleanup or queue advancement · New Drain request handlers before releasing runtime ownership · New Enforce an absolute request-header read deadline · New Reject non-string backend values with a 400 response · New Mark calibration comparison unknown when data is missing · New
15 resolved since last review
Prevent connection reuse when request bodies remain unread Limit concurrent connections and bound socket read times Test runtime token validation and error redaction Publish promotion generations atomically Validate every question and enforce taskset option limits Enforce taskset digest checks for readiness, inference, and rollback Acquire exclusive state-directory ownership before recovery Enforce maxSeconds across training and evaluation Verify restart recovery leaves no candidate or promotion Handle SIGTERM through the cleanup path Move job setup inside the failure-handling try block Preserve rollback history when promoting the current candidate Cancel queued jobs when closing the service Use fixture backends only when backends is None Document direct scoring requests instead of unavailable CLI commands
Review effort: Max (adaptive)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Superseded by a corrected decision-model stack targeting build/hub/new. This PR remains as the reviewed legacy-main implementation history; its host-neutral runtime work will be replayed above the new-Hub decision package port. |



Why
Teams need a safe way to exercise the decision-model lifecycle locally before connecting real training infrastructure. This adds a customer-operated reference that proves jobs, evaluation, promotion, rollback, persistence, and authenticated inference without requiring a GPU or model download.
What changes
The brand-check package gains a local runtime with bounded asynchronous jobs, immutable candidates and evaluation reports, append-only promotion generations, explicit rollback, restart recovery, liveness and readiness endpoints, and an authenticated scoring contract.
A deterministic fixture backend and demo exercise the complete lifecycle. Fixture candidates are marked development/test-only and cannot be promoted, restored, reported ready, or served in production. A small backend interface leaves real-model integration for a later change.
The runtime also includes a dependency-free container image and focused coverage for persistence, exclusive state ownership, candidate integrity, wall-time limits, production gating, request validation, authentication, failure recovery, promotion, rollback, and clean shutdown.
This pull request is stacked on #203.